Numerical Analysis (CS 450) Spring 2026
| What | Where |
|---|---|
| Time/place | Mon/Wed 3:30pm -- 4:45pm 3031 Campus Instructional Facility/CIF / Catalog |
| Class URL | https://bit.ly/cs450-s26 |
| Class recordings | Illinois Mediaspace |
| Discussion | Discuss » (see invite link below to sign up) |
| Administrative Help | Help Desk (click "Message" on the top right) |
| Chat | Chat » |
| Calendar | View » |
| Online Office Hours | Join (only Chicago students may ask questions here) |
Informal Early Feedback
To provide feedback on the course, please click the following button:
If you prefer, you may do this anonymously, e.g., in an "incognito" browser window that is not signed in.
Homework
- Homework set 1 (Due February 4)
- Homework set 2 (Due February 18)
- Homework set 3 (Due February 25)
4-Credit Hour Assignments
(none yet)
Exams
Please find information on our upcoming exams in the corresponding section of the class calendar. Reserve your time slots in the testing facility as soon as possible--otherwise your preferred times may no longer be available.
Lectures
Course Outline
-
Introduction to Scientific Computing
- Notes
- Notes (unfilled, with empty boxes)
- Notes (source code on Github)
- About the Class
- Errors, Conditioning, Accuracy, Stability
- Floating Point
- Demo: Backward Stability by Example
- Demo: Catastrophic Cancellation
- Demo: Conditioning of Evaluating tan
- Demo: Density of Floating Point Numbers
- Demo: Floating Point and the Series for the Exponential Function
- Demo: Floating Point vs Program Logic
- Demo: Floating point and the Harmonic Series
- Demo: Picking apart a floating point number
- Demo: Truncation vs Rounding
- Demo: Vector Norms
- Demo: Writing Testable Numerics Code
-
Systems of Linear Equations
- Theory: Conditioning
- Methods to Solve Systems
- LU: Application and Implementation
- Demo: BLAS Level 2 vs Level 3
- Demo: Coding back-substitution
- Demo: Complexity of Mat-Mat multiplication and LU
- Demo: Condition number visualized
- Demo: Conditioning of 2x2 Matrices
- Demo: LU Factorization with Partial Pivoting
- Demo: LU Factorization
- Demo: Matrix norms
- Demo: Sherman-Morrison
- Demo: Vanilla Gaussian Elimination
-
Linear Least Squares
- Introduction
- Sensitivity and Conditioning
- Solving Least Squares
- Demo: 3x3 Givens demo
- Demo: 3x3 Householder demo
- Demo: Gram-Schmidt and Modified Gram-Schmidt
- Demo: Gram-Schmidt--The Movie
- Demo: Householder in 3D
- Demo: Image compression
- Demo: Interactive Polynomial Fit
- Demo: Issues with the normal equations
- Demo: Keeping track of coefficients in Gram-Schmidt
- Demo: Normal equations vs Pseudoinverse
- Demo: Polynomial fitting with the normal equations
- Demo: Rank-Revealing QR
- Demo: Relative cost of matrix factorizations
-
Eigenvalue Problems
- Properties and Transformations
- Sensitivity
- Computing Eigenvalues
- Krylov Space Methods
- Demo: Arnoldi Iteration
- Demo: Bauer-Fike Eigenvalue Sensitivity Bound
- Demo: Computing the SVD
- Demo: Exploring the Numerical Range
- Demo: Householder Similarity Transforms
- Demo: Motivating Power Iteration
- Demo: Orthogonal Iteration
- Demo: Power Iteration and its Variants
- Demo: QR Iteration
- Demo: Rounding in characteristic polynomial using SymPy
-
Nonlinear Equations
- Introduction
- Iterative Procedures
- Methods in One Dimension
- Methods in $n$ Dimensions (``Systems of Equations'')
- Demo: Bisection Method
- Demo: Convergence of Newton's Method
- Demo: Convergence of the Secant Method
- Demo: Fixed point iteration
- Demo: Newton's Method
- Demo: Newton's method in n dimensions
- Demo: Rates of Convergence
- Demo: Secant Method
- Demo: Three quadratic functions
-
Optimization
- Introduction
- Methods for unconstrained opt. in one dimension
- Methods for unconstrained opt. in $n$ dimensions
- Nonlinear Least Squares
- Constrained Optimization
- Demo: Conjugate Gradient Method
- Demo: Gauss-Newton
- Demo: Nelder-Mead Method
- Demo: Newton's Method in 1D
- Demo: Newton's Method in n dimensions
- Demo: Sequential Quadratic Programming
- Demo: Steepest Descent
-
Interpolation
- Introduction
- Methods
- Error Estimation
- Piecewise interpolation, Splines
- Demo: Chebyshev Interpolation
- Demo: Choice of Nodes for Polynomial Interpolation
- Demo: Composite Gauss Interpolation Error
- Demo: Interpolation Error
- Demo: Interpolation with Radial Basis Functions
- Demo: Jump with Chebyshev Nodes
- Demo: Lebesgue Constant
- Demo: Monomial interpolation
- Demo: Orthogonal Polynomials
- Demo: Playing with Barycentric Interpolation
-
Numerical Integration and Differentiation
- Numerical Integration
- Quadrature Methods
- Accuracy and Stability
- Gaussian Quadrature
- Composite Quadrature
- Numerical Differentiation
- Richardson Extrapolation
- Demo: Accuracy of Newton-Cotes
- Demo: Finite Differences vs Noise
- Demo: Floating point vs Finite Differences
- Demo: Gaussian quadrature weight finder
- Demo: Newton-Cotes weight finder
- Demo: Richardson with Finite Differences
- Demo: Taking Derivatives with Vandermonde Matrices
-
Initial Value Problems for ODEs
- Setup
- Existence, Uniqueness, Conditioning
- Numerical Methods (I)
- Accuracy and Stability
- Stiffness
- Numerical Methods (II)
- Demo: Backward Euler stability
- Demo: Dissipation in Runge-Kutta Methods
- Demo: Forward Euler stability
- Demo: Predator-Prey System
- Demo: Stability regions
- Demo: Stiffness
-
Boundary Value Problems for ODEs
- Existence, Uniqueness, Conditioning
- Numerical Methods
- Demo: Finite differences
- Demo: Shooting method
- Demo: Sparse matrices
- Partial Differential Equations and Sparse Linear Algebra
-
Fast Fourier Transform
- Develop this
- Random Numbers and Simulation
- Additional Topics
- Wrap-up
CAUTION!
These scribbled PDFs are an unedited reflection of what I wrote during class. They need to be viewed in the context of the class discussion that led to them. See the lecture videos for that.
If you would like actual, self-contained class notes, look in the outline above.
These scribbles are provided here to provide a record of our class discussion, to be used in perhaps the following ways:
- as a way to cross-check your own notes
- to look up a formula that you know was shown in a certain class
- to remind yourself of what exactly was covered on a given day
By continuing to read them, you acknowledge that these files are provided as supplementary material on an as-is basis.
Team
(Instructor)
Email: andreask@illinois.edu
Office: 4318 (Office hours in the Siebel basement tutoring space) Siebel
Samuel Grayson
(TA)
Email: grayson5@illinois.edu
Office: CS Tutoring Space in Siebel basement Siebel
Statement on CS CARES, Values, and Code of Conduct
All members of the Illinois Computer Science department---faculty, staff, and students---are expected to adhere to the CS Values and Code of Conduct. The CS CARES Committee is available to serve as a resource to help people who are concerned about or experience a potential violation of the Code. If you experience such issues, please contact the CS CARES Committee. The instructor of this course are also available for issues related to this class.
Textbook
Scientific Computing: An Introductory Survey / E-Book (accessible free of charge from campus network/VPN)
Michael T. Heath, Revised Second Edition, Society for Industrial and Applied Mathematics
Computing
We will be using Python with the libraries numpy, scipy and matplotlib for in-class work and assignments. No other languages are permitted. Python has a very gentle learning curve, so you should feel at home even if you've never done any work in Python.
Running Code on your Own Computer
While running code in this online system should technically suffice to do your work for this class, you may find it useful to also install Python on your own computer.
The recommended way of doing so involves downloading the Anaconda Python distribution. Note that this is a commercial product (even if it is free of charge), and this is not intended as an endorsement of the company or the product. Note that we cannot promise to provide technical support for this installation.
Another way to run Python code is through an online JupyterLab available through the course. Go to https://relate.cs.illinois.edu/lab get started. NOTE that this environment runs entirely in your browser. If you clear your browser data, any work 'saved' there will be irretrievably lost.
Grading Policies
Mental Health
Significant stress, mood changes, excessive worry, substance/alcohol misuse or interferences in eating or sleep can have an impact on academic performance, social development, and emotional wellbeing. The University of Illinois offers a variety of confidential services including individual and group counseling, crisis intervention, psychiatric services, and specialized screenings which are covered through the Health Service Fee. If you or someone you know experiences any of the above mental health concerns, it is strongly encouraged to contact or visit any of the University’s resources provided below. Getting help is a smart and courageous thing to do for yourself and for those who care about you.
- Counseling Center (217) 333-3704
- McKinley Health Center (217) 333-2700
- National Suicide Prevention Lifeline (800) 273-8255
- Rosecrance Crisis Line (217) 359-4141 (available 24/7, 365 days a year)
- If you are in immediate danger, call 911.
This statement is approved by the University of Illinois Counseling Center.
Python Help
- Python tutorial
- Facts and myths about Python names and values
- Learn Python the hard way
- Project Euler (Lots of practice problems)
- PythonTutor (Execute Python step-by-step, with pictures)
Numpy Help
(see section 1 of the outline for more)
- From Python to Numpy
- With associated 100 numpy exercises
- The SciPy lectures
- The Numpy User Guide
- More in this reddit thread
- An introduction to Numpy and SciPy